Measure Concentration of Hidden Markov Processes
نویسنده
چکیده
We prove what appears to be the first concentration of measure result for hidden Markov processes. Our bound is stated in terms of the contraction coefficients of the underlying Markov process, and strictly generalizes the Markov process concentration results of Marton (1996) and Samson (2000). Somewhat surprisingly, the hidden Markov process is at least as “concentrated” as its underlying Markov process; this property, however, fails for general hidden/observed process pairs.
منابع مشابه
Relative Entropy Rate between a Markov Chain and Its Corresponding Hidden Markov Chain
In this paper we study the relative entropy rate between a homogeneous Markov chain and a hidden Markov chain defined by observing the output of a discrete stochastic channel whose input is the finite state space homogeneous stationary Markov chain. For this purpose, we obtain the relative entropy between two finite subsequences of above mentioned chains with the help of the definition of...
متن کاملObtaining measure concentration from Markov contraction
Concentration bounds for non-product, non-Haar measures are fairly recent: the first such result was obtained for contracting Markov chains by Marton in 1996. Since then, several other such results have been proved; with few exceptions, these rely on coupling techniques. Though coupling is of unquestionable utility as a theoretical tool, it appears to have some limitations. Coupling has yet to ...
متن کاملMeasure Concentration of Strongly Mixing Processes with Applications
The concentration of measure phenomenon was first discovered in the 1930’s by Paul Lévy and has been investigated since then, with increasing intensity in recent decades. The probabilitytheoretic results have been gradually percolating throughout the mathematical community, finding applications in Banach space geometry, analysis of algorithms, statistics and machine learning. There are several ...
متن کاملSpeech enhancement based on hidden Markov model using sparse code shrinkage
This paper presents a new hidden Markov model-based (HMM-based) speech enhancement framework based on the independent component analysis (ICA). We propose analytical procedures for training clean speech and noise models by the Baum re-estimation algorithm and present a Maximum a posterior (MAP) estimator based on Laplace-Gaussian (for clean speech and noise respectively) combination in the HMM ...
متن کاملConcentration of Measure on Product Spaces with Applications to Markov Processes
Abstract. For a stochastic process with state space some Polish space, this paper gives sufficient conditions on the initial and conditional distributions for the joint law to satisfy Gaussian concentration inequalities and transportation inequalities. In the case of the Euclidean space Rm, there are sufficient conditions for the joint law to satisfy a logarithmic Sobolev inequality. In several...
متن کامل